Hi @jjmachan, can you help me with this one, all the unit test have failed because of the PR #1879

The code changed was small
From this

def validate_samples(self, samples: t.List[Sample]) -> t.List[Sample]:
    """Validates that all samples are of the same type."""
    if len(samples) == 0:
        return samples

    first_sample_type = type(self.samples[0])
    if not all(isinstance(sample, first_sample_type) for sample in self.samples):
        raise ValueError("All samples must be of the same type")

    return samples

to this

def validate_samples(self, samples: t.List[Sample]) -> t.List[Sample]:
    """Validates that all samples are of the same type."""
    if len(samples) == 0:
        return samples

    first_sample_type = type(samples[0])
    for i, sample in enumerate(samples):
        if not isinstance(sample, first_sample_type):
            raise ValueError(f"Sample at index {i} is of type {type(sample)}, expected {first_sample_type}")

    return samples

which does not change much of the functionality of comparing classes while iterating

Hi @jjmachan, does this solution align with with the given issue?

Hi @jjmachan, updated the traces functionality to have {prompt_number} : {trace},

New traces should look like

{'prompt_1': {'name': 'claim_decomposition_prompt', 'input': ClaimDecompositionInput(response='Eifel tower is in Paris'), 'output': ClaimDecompositionOutput(claims=['Eiffel tower is in Paris.'])}, 'prompt_2': {'name': 'n_l_i_statement_prompt', 'input': NLIStatementInput(context='Paris, France is a city where Eifel tower is located', statements=['Eiffel tower is in Paris.']), 'output': NLIStatementOutput(statements=[StatementFaithfulnessAnswer(statement='Eiffel tower is in Paris.', reason='The context explicitly states that Eiffel tower is located in Paris, France.', verdict=1)])}, 'prompt_3': {'name': 'claim_decomposition_prompt', 'input': ClaimDecompositionInput(response='Paris, France is a city where Eifel tower is located'), 'output': ClaimDecompositionOutput(claims=['Paris is a city in France.', 'Eiffel Tower is located in Paris.'])}, 'prompt_4': {'name': 'n_l_i_statement_prompt', 'input': NLIStatementInput(context='Eifel tower is in Paris', statements=['Paris is a city in France.', 'Eiffel Tower is located in Paris.']), 'output': NLIStatementOutput(statements=[StatementFaithfulnessAnswer(statement='Paris is a city in France.', reason='The context does not provide any information about the location of Paris.', verdict=0), StatementFaithfulnessAnswer(statement='Eiffel Tower is located in Paris.', reason='The context explicitly states that the Eiffel Tower is in Paris.', verdict=1)])}}

Hi @sahusiddharth, can you check, is this because of the previous unit test PR you have made?

hey @Vidit-Ostwal this is great - much for readable

but one concern is that this is too big of a change and might break - can I keep this on pause till v0.3?

we are planning to maybe release that with another feature we wanted. This way we keep the honor semver best

@sahusiddharth also roping you in on this

@jjmachan, sure.
Can coordinate with @sahusiddharth as well, to resolve some conflicts.